来自现有RDD的java Spark流媒体

Question

任何人请帮助我如何从现有的RDD创建数据流。 我的代码是：

JavaSparkContext ctx = new JavaSparkContext(conf);
JavaRDD<String> rddd = ctx.parallelize(arraylist);

现在我需要使用这些rddd作为JavaStreamingContext的输入

Answer 1

试试queueStreamAPI
RDD队列作为一个流，推送到队列中的每个RDD都将被视为数据流中的一批数据，并像流一样进行处理

public <T> InputDStream<T> queueStream(scala.collection.mutable.Queue<RDD<T>> queue,
                              boolean oneAtATime,
                              scala.reflect.ClassTag<T> evidence$15)

Create an input stream from a queue of RDDs. In each batch, it will process either one or all of the RDDs returned by the queue.
NOTE: Arbitrary RDDs can be added to queueStream, there is no way to recover data of those RDDs, so queueStream doesn't support checkpointing.